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DEFINING AND MEASURING THE SUCCESS 
OF SERVICE CONTRACTS 


ABSTRACT 


Services acquisition in the U.S. Department of Defense (DoD) has continued to increase 
in scope and dollars in the past decade. The DoD has spent more on services than on 
supplies, equipment and goods together, totaling approximately 57% of total acquisition 
expenditures and nearly a third of the total DoD budget. As a result, the agency must 
give greater attention to the management of services acquisition. Stakeholder theory 
illustrates how acquisition team members often have conflicting goals and objectives, 
leading to differing definitions and measurements of a successful service contact. We 
used stakeholder theory to address the following questions: (1) how are successful service 
contracts within the DoD being defined by different stakeholders, (2) how are service 
contracts being measured within the DoD by different stakeholders, (3) how should 
service contracts be defined and measured within the DoD. We conducted 41 interviews 
and surveys of key stakeholders. Our findings reveal no standardized definition or 
measurements for success of service contracts. However, some salient characterstics of 
definitions are schedule, maintain costs, and well defined requirements. With respect to 
measurements, relevant characteristics included performance and cost. Furthermore, we 
provide recommendations on establishing standardized definitions and measurements of 


SsuCCeSS. 
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I. INTRODUCTION 


A. BACKGROUND 


Over the last few decades, a growing trend of increased dollar value in service 
contracts for Department of Defense (DoD) contracting has become apparent. Relative to 
supply contracts, services acquisition has continued to grow both in terms of dollar value 
and in range of acquisitions. This trend is shown in Figure 1, Department of Defense 
contracts for goods and services (Rendon, Apte, & Apte, 2012). This demonstrates that 
between fiscal year (FY) 2000 and FY2010, growth in service contracts more than 
doubled. Contract obligations rose to over $387 billion in 2008, with nearly $200 billion 
spent on services alone (Hutton & Solis, 2009). As such, a management and oversight 


plan and clearly defined metrics for successful service contracts are important. 
FIGURE 1. DoD’S CONTRACTS FOR GOODS AND SERVICES 
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Figure 1. DoD Contracts for Goods and Services 
(From Rendon, Apte, & Apte, 2012) 


Figure 2 clearly demonstrates this growing trend in service contract expenditures. 


Funding spent on service contracts grew steadily from 1990-2010, constituting roughly 
1 


42% of the total spending on contracts by the DoD, exclusive of research and 
development services contracts. Notably, service contracts showed the highest growth in 
percentage of expenditures over the last 21 years, with a rise of nearly 6.1% annually 
(Ellman, Livergood, Morrow, & Sanders, 2011). An in-depth study of services 
acquisition will help develop recommendations to evaluate the factors of success in 


services contracting. 
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Figure 2. Growth in DoD Service Contract Expenditures 
(From Ellman et al., 2011) 


The definition and measurement of successful service contracts should align with 
the overarching initiatives, as illustrated by the Under Secretary of Defense for 
Acquisition, Technology, and Logistics (USD[AT&L]). The USD(AT&L) outlined these 
initiatives in his June 2010 memorandum on acquisition efficiency. In his memorandum, 


2 


he noted that the Department of Defense currently spends nearly $400 billion out of its 
$700 billion budget on contracted goods and services. The DoD has set a goal to achieve 
savings of more than $100 billion over the next five fiscal years. The USD(AT&L) 
describes this initiative in terms of an efficiency and cost-effectiveness overhaul. The 
DoD, according to the USD(AT&L), has the potential of increasing its warfighter 
capability by 2—3% annually, without future budget increases. In addition, identifying 
and cutting unproductive or low-value programs and contracts will free up funding to 
transfer to more productive warfighter programs. The USD(AT&L) states that the United 
States is entering a period of budget reform, stunting the budget growth of the previous 
decade. The ability to properly manage and access every service contract is essential to 
reduce inefficiencies and eliminate nonperformance, and, consequentially, to achieve the 


level of savings required by the DoD savings initiative (USD[AT&L], 2010). 


B. PURPOSE 


Our purpose with this research was to conduct a comprehensive investigation into 
the definition and metrics of a successful service contract from the perspective of various 
stakeholders. |Our data collection methods used interviews conducted among 
stakeholders within the DoD. With our research we determine if the stakeholders define 
and measure the success of service contracts differently. The results of this project 
support ongoing research being conducted by the Acquisition Research Program at the 
Naval Postgraduate School concerning the DoD’s management of service contracts 


(Apte, Apte & Rendon, 2010). 


C; RESEARCH QUESTIONS 


With our research we attempt to answer the following questions as they relate to 
the definition and measurement of successful service contracts within the DoD, and as 


determined by the different identified stakeholders: 


1. How are successful service contracts within the DoD being defined by 


different stakeholders? 


2: How are service contracts being measured within the DoD by different 


stakeholders? 


3. How should service contracts be defined and measured within the DoD? 


D. BENEFITS AND LIMITATIONS OF THIS RESEARCH 


In addition to identifying how stakeholders currently define and measure the 
success of service contracts, we aim to provide key information to develop sound 
protocol and metrics for future service contract success. By determining how each 
stakeholder within the DoD defines a successful service contract, we endeavor to 
distinguish the key driving factors of service contract success that lead to greater 


performance and savings. 


This research was limited by the sample size and scope of the population of 
stakeholders interviewed. Of the numerous organizations within the DoD, a small 


percentage of these stakeholders participated in the research. 


E. RESEARCH METHODOLOGY 


This research utilized a web-based survey, telephone interviews, and personal 
interviews of the various stakeholders within DoD service contract activities. The survey 
consisted of two open-ended questions and three demographic questions. We conducted 
a review of literature on service contract management and stakeholder theory. We then 
developed a survey to investigate the definition and measurement of successful service 
contracts. The survey was deployed across three contracting commands within the Navy. 
The responses of all participants were analyzed and examined for differences and 
commonality. We then developed conclusions and provided recommendations for the 
definition and measurement of a successful service contract process. Finally, we 
analyzed and examined the results to identify and categorize how stakeholders define and 


measure successful service contracts. 


F. ORGANIZATION OF REPORT 


This report is organized into five chapters. In Chapter I, we include background 
information, the purpose of the research, our research questions, the benefits and 
limitations of the research, and the research methodology. In Chapter II, we review past 
and current literature on the services contracting process. We describe the members of 
the acquisition team, their roles and responsibilities, as well as their goals and objectives. 
We then identify the deficiencies in service contract management evidenced through 
several Government Accountability Office (GAO) and Inspector General (IG) findings. 
Further, we present the research on stakeholder theory and how it relates to the service 
contract management process. In Chapter II, we outline the research methodology, 
which includes our data collection and analytical process. In Chapter IV, we examine 
and analyze the research findings. In Chapter V, we provide the summary, conclusions, 


and areas for further research. 


G. SUMMARY 


In this chapter, we provided background information on service acquisition within 
the DoD, the purpose of the report, our research questions, the benefits and limitations of 
our research, and the methodology and organization of the report. The information we 
provided outlined the objectives described by the USD(AT&L) and how these objectives 
relate to the definition and measurement of successful service contracts. The research 
questions are the primary focus of this report. In Chapter II, we review past and current 
literature on the service contracting process, the acquisition team, roles and 
responsibilities of service acquisition personnel, goals and objectives of service 


acquisition personnel, deficiencies in service contracts, and stakeholder theory. 
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I. LITERATURE REVIEW 


A. INTRODUCTION 


In Chapter I, we established the foundation of this research. In Chapter II, we 


introduce past and current literature on the service contracting process, the acquisition 


team, roles and responsibilities of service acquisition personnel, goals and objectives of 


service acquisition personnel, deficiencies in service contracts, and stakeholder theory. 


B. SERVICE CONTRACTING PROCESS 


Service contract management is defined as the art and science of managing an 


agreement throughout the process of contracting (Rendon & Snider, 2008). The contract 


management process can be described utilizing a six-step model. Following is the list of 


phases within the process and the key activities of each phase: 


1. 


Procurement Planning: Identification of which organization or business 
needs can be best achieved by procurement of services or products 
external to the organization. Key activities include determining the scope 
of work, completing market research, technology analysis, and funding 
determination, and creating estimates for cost and schedule. 

Solicitation Planning: Preparation of solicitation documentation to support 
the acquisition. Key activities include using standardized forms, model 
contracts, specifications and descriptions of items, and terms and 
conditions of the contract. 

Solicitation: Obtaining bids and proposals from prospective contractors on 
how to meet the objectives of the service contract. Pertinent activities 
include conducting advertising to identify new sources and compiling a 
list of interested offerors. 

Source Selection: Receipt of proposed bids and application of selection 


criteria for supplier products or services. Activities specific to this phase 


include using evaluation criteria centered on management, technical, and 
cost factors and considering an offeror’s past performance in evaluating 
proposals. 

s. Contract Administration: Ensuring that each supplier’s performance is in 
accordance with contractual requirements. Key activities include 
employing an integrated team approach to monitor the contractor’s cost, 
schedule, and performance. Additionally, this phase includes establishing 
a process for administering incentives for award fee provisions. 

6. Contract Closeout: Validation of administrative matters pertaining to 
completed contracts. Activities unique to this phase includes using 
checklists and forms for documentation of closed contracts and 
maintaining lessons learned and best practices for use in future contracts 


(Rendon & Garrett, 2005). 


The outlined activities and steps in the service contract process are performed by 


the various members of the acquisition team, as discussed in the next section. 


C. THE ACQUISITION TEAM 


Throughout the federal acquisition service contracting process, the vision is to 
deliver the best value to the customer in a timely manner, while maintaining the public’s 
trust and fulfilling public policy objectives (FAR, 2012). The Federal Acquisition 
Regulation (FAR) further delineates that all the participants in the acquisition process 
should work together as a team and should be empowered to make decisions within their 
area of responsibility (FAR, 2012). The FAR defines the acquisition team as consisting 
of all participants in government acquisition, including not only representatives of the 
technical, supply, and procurement communities but also the customers they serve and 
the contractors who provide the products and services (Federal Acquisition Regulation, 
2012). According to the FAR, the role of each member of the acquisition team is to 
exercise personal initiative and sound business judgment in providing the best value 


product or service to meet the customer’s needs (Federal Acquisition Regulation, 2012). 


These members can be senior agency leaders, government personnel, administrative 


employees, and even support employees (U.S. Merit Systems Protection Board, 2005). 


Many commercial organizational structures have incorporated the use of cross- 
functional teams in an effort to improve communication, coordination, and collaboration 
among the team (Monczka, Handfield, Giunipero, & Patterson, 2011). The DoD 
implements the cross functional team through the use of an Integrated Product Team 
(IPT). It is useful to note that the IPT is primarily used in contracting for products to 
facilitate the process of meeting cost, performance, and schedule objectives from product 
concept through production, including field support (Press, 2012). In service contracting 
the IPT is a team composed of representatives from appropriate functional disciplines 
working together to identify and resolve issues; make sound, timely recommendations in 
an effort to facilitate decision-making; and build successful programs that meet the 


warfighter’s needs (Press, 2012). 


This research examines the internal members who could be considered 
stakeholders of the acquisition, specifically the program manager (PM), principal 


contracting officer (PCO), and contracting officer representative (COR). 


1, Project Manager 


The project management profession is principally represented by the Project 
Management Institute (PMI) and the International Project Management Association 
(IPMA); each group has its own representative professional certification (Garrett, 2010). 
Project management is defined as the centralized, coordinated management of a program 
to achieve the program’s strategic objectives and benefits (Project Management Institute, 
2008). For the purposes for this report, the project manager is termed program manager 
(PM). The PM has the ultimate responsibility for all cost, schedule, and technical aspects 
of the program (Deneault & Stambaugh, 2000, p. 22). A PM’s responsibilities cut across 
multiple acquisition functional areas (e.g., business, contracting, facilities engineering, 
information technology, life cycle logistics, quality, systems planning) as well as 
knowledge in other technical areas (Krieger, 2011la, p. 24). In most companies, PMs 


serve as multifunctional team leaders on one or more projects, responsible for achieving 
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the desired results for the projects (Garrett, 2010). However, most PMs lack the authority 
to sign, modify, or cancel contracts that legally bind companies (Garrett, 2010). The 
same is true for DoD PMs. In the DoD, the PM is responsible for identifying the 
requirements, managing the project processes, planning, monitoring, and executing the 
project. Further, the PM is accountable for addressing the various needs, concerns, and 
expectations of the stakeholders as the project is planned and executed (Project 
Management Institute, 2008). In carrying out the role of PM, various constraints must be 
balanced. The most common constraints are the following: scope, quality, schedule, 
budget, resources, and risk (Project Management Institute, 2008). These are often 
competing constraints as the project progresses. Scope is defined as the work that needs 
to be accomplished to deliver the service. Included in scope is the management plan, the 
description of how the project scope will be managed and controlled as well as a baseline 
that is compared to the actual results. Quality is the results toward which the constraints 
are driving. Schedule is the sequence of activities and duration to complete the service. 
Budget is managing, estimating and controlling costs of the service. Resources refer to 
the human personnel constraint that must be managed. In all projects, risk must be 
identified, monitored, and managed. All of these common constraints and their 
interdependencies are best illustrated in the project management triangle shown in Figure 
3. Each side represents a constraint. These three constraints are often competing 
constraints: increased scope typically means increased time and increased cost, a tight 
time constraint could mean increased costs and reduced scope, and a tight budget could 
mean increased time and reduced scope (Sekhar, 2010). The DoD 5000 Series is a 
regulatory document that provides guidance and policy for the management of defense 
acquisitions. Program management is the management of all of the project goals and 
objectives, while honoring the preconceived constraints as well as ensuring compliance 


with the 5000 Series directives in the defense acquisition guidebook (Krieger, 201 1a). 
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Figure 3. Project Management Triangle 


(From Font, V., 2010 ) 


According to A Guide to Best Practices for Contract Administration, program 
management activities include planning, organizing, securing, and managing resources to 
achieve specific goals (Office of Federal Procurement Policy, 1994). The PM adapts to 
various internal procedures of the contracting process and forms close links with the 
various stakeholders. To achieve client satisfaction, it is essential that the PM realize key 
issues of cost, schedule, and performance. In government acquisition, the PM should 
obtain integrated cost and schedule performance data at an appropriate level in order to 
monitor program execution. The PM should require contractors and government 


activities to use internal management control systems that accomplish the following: 


e relate time-phased budgets to specific tasks identified in the statement of 
work; 

e produce data that indicate work progress; 

e properly relate cost, schedule, and technical accomplishment; and 


e produce data that are valid, timely, and auditable. (USD[AT&L]), 2008) 


In government acquisition the PM is ultimately accountable to the end user and 
Congress. Therefore, the PM activities include extensive coordination with internal and 
external stakeholders (Deneault & Stambaugh, 2000, p. 23). The PM owns the 
acquisition strategy which drives the acquisition plan (Krieger, 201la, p. 42). This 
strategy shall be in writing and prepared in accordance with all the requirements of 


subpart 7.1. of the FAR (2012). The strategy is the PM’s overall plan for satisfying the 
11 


mission need in the most effective, economical, and timely manner, so before there is a 
contract, the PM owns the acquisition strategy that the contracting officer will seek to 


implement (Krieger, 201 1a, p. 42). 


In government acquisition, the PM is seen as successful if they continue to 
perpetuate their program through the acquisition life cycle. This focus tends to take 
precedence over reporting realistic program status. PMs are, by nature, becoming 
program advocates. A PMs’ future assignments and promotions depend on the success of 
the program, and it becomes very difficult for the PM to “blow the whistle,” hoping that 
they will be transferred before the true costs of the project become known (Fox, Hirsch, 
Krikorian, & Schumacher, 1994). The PM receives guidance and oversight from the all 
three branches of the government; serious problems with a program can significantly 
jeopardize congressional and Office of the Secretary of Defense support for the program 
(Krieger, 201la). PMs sometimes lack realism and have undue optimism; a program 
cancellation or reduction in scope is perceived as a PM’s failure by their service 
superiors. Therefore, a conflict exists between reporting realistic program status and 
doing what is necessary to keep programs funded and moving through the acquisition life 


cycle (Fox et al., 1994). 


p- Principal Contracting Officer 


The contract management profession is represented by three professional 
associations: the National Contract Management Association (NCMA), the Institute for 
Supply Management (ISM), and the International Association of Contract and 
Commercial Management (IACCM). Each group has its own professional certifications 
(Garrett, 2010). Contract management is a career field that takes significant on-the-job 
training to become truly competent (Deneault & Stambaugh, 2000, p. 21). Only a few 
educational institutions offer degrees or professional certificates in government contract 
management, and education does not always equate to experience (Deneault & 
Stambaugh, 2000, p. 21) A contract manager must be able to integrate functional inputs 
into a solicitation and the resulting contract (Krieger, 2011b, p. 25). Despite the fact 
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contract managers are seldom responsible for daily project planning or operation, they are 
authorized to enter their organizations into legally binding contractual arrangements 
(Garrett, 2010). Contract managers must be able to examine the contractual meaning of 
pre-contractual events and documents. Contract managers must discern the objectives, 
needs, limitations, and even prejudices of other stakeholders (Hirsch, 1986). When a 
contract manager has this sensitivity, the contract manager can reduce strife and gain 
empathy towards other members in the acquisition team (Hirsch, 1986). A contract 
manager must have the skills to answer the following questions: (1) What is the work? 
(2) What are and where do we find the skills to perform the tasks? (3) How do we ensure 
the work is done effectively at the lowest price (Cohen & Eimicke, 2008)? Most 
organizations empower one or more employees in the role of contract management, and 
in some agencies a relatively small number of high-level officials are designated 
contracting managers solely by virtue of their positions (National Archives and Records 
Administration, 2012). This employee (the contract manager) is referred to in contract 
law as the agent; the source of authority is referred to as the principal (Garrett, 2010). In 
DoD contracting this agent is called the principal contracting officer (PCO) and the 
principal is the federal government. PCOs have the sole authority of the government to 
legally bind, enter into, administer, and terminate contracts and make related 
determinations and findings. PCOs may bind the government only to the extent of the 
authority delegated to them in clear, written instructions as to the limits of their authority 
through their warrant (Federal Acquisition Regulation, 2012). The PCO is responsible 
for meeting the conditions of the contract, and therefore spends more time on business 
and legal issues and contract administration (Deneault & Stambaugh, 2000, p. 23). A 
PCO’s warrant is taken very seriously; it distinguishes an individual for taking on a 
significant amount of responsibility and accountability. 

Warrants are not taken for granted after they are received because they can 

be lost through job changes or the belief by superiors that a person lacks 

the skills or knowledge necessary to uphold the position. When a warrant 


is lost a contracting officer has no choice but to change careers or earn 
another warrant. (Deneault & Stambaugh, 2000, p. 22) 
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The government’s warrant authorizes the PCO as the only individual who can financially 
obligate the organization and requires them to be involved in all communications 
(Deneault & Stambaugh, 2000, p. 22). Thus, a significant amount of training is required 
on the constraints of the law and requirements of the contract (Deneault & Stambaugh, 
2000, p. 22). No contract shall be entered into unless the PCO ensures that all 
requirements of law, executive orders, regulations, and all other applicable procedures, 
including clearances and approvals, have been met (Federal Acquisition Regulation, 
2012). Further, the PCO is responsible for ensuring performance of all necessary actions 
for effective contracting, to include ensuring compliance with the terms of the contract 
and safeguarding the interests of the United States in its contractual relationships while 
ensuring adherence to procurement laws and regulations, given their ability and 
instructed by Title 48 of the Code of Federal Regulations (CFR), the FAR (2012), and the 
Defense Federal Acquisition Regulations Supplement (DFARS; 2010). Additionally, the 
PCO is responsible for assisting in clarifying agency needs, market research, contract 
methods, acquisition planning, competing and meeting source selection requirements, and 
conducting negotiations (Hirsch, 1986). In order to perform these responsibilities, 
contracting officers should be allowed wide latitude to exercise business judgment. The 
FAR specifically mandates that contracting officers shall 
(a) Ensure that the requirements of 1.602-1(b) (FAR 2012) have been met, 
and that sufficient funds are available for obligation; 
(b) Ensure that contractors receive impartial, fair, and equitable treatment; and 
(c) Request and consider the advice of specialists in audit, law, engineering, 
information security, transportation, and other fields, as appropriate. 
(d) Unless the contracting officer retains and executes the COR duties, in 
accordance with agency procedures, designate and authorize, in writing, a 
COR on all contracts and orders other than those that are firm-fixed price, 
and for firm-fixed-price contracts and orders as appropriate (Federal 


Acquisition Regulation, 2012). 
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3. Project Manager and Principal Contracting Officer Similarities and 
Differences 


Often PMs and PCOs work in a matrix organization characterized by 
multifunctional teams. PMs’ and PCOs’ roles often overlap in terms of competencies and 
responsibilities. This is evident in looking at their professional certification programs, 
bodies of knowledge, and day-to-day interactions (Garrett, 2010). On the acquisition 
team, the PM is responsible for what needs to be done to execute the program through the 
phases outlined in the acquisition strategy; the PCO and the other members of the 
acquisition team implement the strategy through the contract (Krieger, 2011b, p. 42). In 
successful programs PMs and PCOs work together effectively, yet there is a great deal of 
conflict. In some instances, the PM is separated from the supported team members 
(Deneault & Stambaugh, 2000, p. 22). PMs rely on their functional departmental support 
to achieve program success. In this interaction, there is a potential for conflicts between 
the stakeholders’ interests; the PM is overall in charge, but the PCO is the only member 
that can challenge the PMs authority (Deneault & Stambaugh, 2000, p. 22). PMs often 
feel that their flexibility is constrained by the conservative interpretations of the PCO 
(Deneault & Stambaugh, 2000). PM and PCO roles overlap at the intersection where 
program execution meets contracting officers’ authority. Flexibility and adaptability have 
become prerequisite traits for PMs and PCOs to succeed (Deneault & Stambaugh, 2000, 
p. 22). 


As previously discussed, the PM is focused on meeting cost, schedule, and 
performance objectives while progressing through the project life cycle. The PCO is 
responsible for ensuring all aspects of the law are complied with and protecting the 
government’s interests. These two different perspectives are often in conflict with each 
other. The PCO’s objectives do not align with the PM’s. This could be the reason why 
one could say the PCO is the unpopular player on the acquisition team (Hirsch, 1986). 


4. Contracting Officer Representative 


PCOs designate and authorize the CORs. As the technical subject matter expert 


on the acquisition team, the COR is an integral stakeholder in the contracting process and 
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is the first line of surveillance on government contracts. The COR also plays a critical 
role in contract administration. Agencies and departments have many different titles to 
describe the COR. Other titles used for this role are government technical representative 
(GTR) and government technical evaluator (GTE; Office of Federal Procurement Policy, 
1994). For purposes of this thesis, we use COR because it is the most common title for 
this function. The COR is nominated by the PCO, as early as practicable in the 
acquisition process (FAR, 2012). The COR’s administrative duties range from simple to 
complex, dependent on the type of contract, contractor performance, and the nature of the 
work. The COR functions as the “eyes and ears” of the contracting officer and monitors 
technical performance, reporting any potential or actual problems to the contracting 
officer (Office of Federal Procurement Policy, 1994). A COR must stay in close 
communication with the contracting officer, relaying any information that may affect 
contractual commitments and requirements. The FAR (2012) specifically mandates that 


a COR must meet the following criteria: 


e shall be a government employee, unless otherwise authorized in agency 
regulations; 

@ shall be certified and maintain certification; 

e must be qualified; 

e may not re-delegate responsibility to perform functions that have been 
delegated; 

e has no authority to make any commitments or changes that affect price, 


quality, quantity, delivery, or other terms and conditions of the contract; 


e nominated by the requirements official. 


It should be noted that although the COR is a critical member of the acquisition 
team as defined by the FAR, the COR is not a member of the acquisition work force, as 


defined by the Defense Acquisition Work Force Improvement Act (DAWIA; 1990). 


The COR provides the technical expertise necessary for successful contracting 
and plays a critical role in affecting the outcome of the contract administration process, as 
well as ensuring maximum return on contract dollars. The following is an example taken 


from the researchers own experience of COR duties for a consulting services contract: 
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Control all government technical interfaces with the contractor. 


Ensure that a copy of all government technical correspondence is 
forwarded to the contracting officer for placement in the contract 


(delivery/task order) file. 


Promptly furnish documentation on any requests for change, deviation, or 
waiver, whether generated by the government or the contracting officer 


(and ordering officer) for their action. 


Determine causes when the contract is not progressing as expected and 


make recommendations to the contracting officer for corrective action. 


Monitor contractor performance to ensure individual contractor employees 
are of the skill levels required and are actually performing at the levels 


charged against the contract during the performance period. 


Monitor contractor performance to ensure that the labor hours charged 
against the contract are consistent and reasonable for the effort completed 


and that any travel charged was necessary and actually occurred. 


Monitor government furnished property. Ensure that property provided to 


the contractor is authorized by the contract. 


Complete the COR Report of Contractor’s Performance in accordance 
with the schedule established in the contract administration plan for a 


contract. 


The PM, PCO, and COR are critical stakeholders responsible for overall program 


These stakeholders hold key decision-making positions and at times may 


overexert their influence based on their positions, resulting in difficult and contentious 


conflicts (Deneault, & Stambaugh, 2000). 


As previously discussed, each stakeholder has conflicting goals and objectives, as 


well as different guiding policies and directives. These all could lead to deficiencies in 


DoD services contracting. We will present these deficiencies in the next section. 
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D. DEFICIENCIES IN SERVICE CONTRACTS 


The DoD’s contract obligations have doubled between fiscal years 2001 and 2008 
to over $387 billion with $200 billion expended on services (Government Accountability 
Office [GAO], 2009). Commensurately, the acquisition workforce declined from 
500,000 to 200,000 personnel in 2006 (Gansler, 2011, p. 237). The downsizing of the 
defense acquisition workforce has reduced a qualified contracting and acquisition 
workforce necessary to manage the increased service contract workload (GAO, 2002b, 
2009a). Human capital problems are debilitating many agencies and threaten the ability 
of others to perform their missions efficiently and effectively (GAO, 2001). Both the 
GAO and the DoD Inspector General (IG) have indicated that failing to maintain an 
adequate workforce to manage the billion-dollar acquisitions increases the risk of poor 
acquisition outcomes and the likelihood of fraud, waste, and abuse (GAO, 2009. From 
2001 to 2009 the GAO has issued 16 reports identifying deficiencies, trends, and 
challenges in contract management. In addition, the DoD IG issued 142 reports on 
deficiencies in the DoD acquisition and contract administration process. Deficiencies 
include selection of an inappropriate contract type allocating unnecessary risk to the 
government. DoD contracting officers have selected risk-laden cost contracts for services 
in which a fixed contract type could have been used, diverting the majority of the risk to 
the contractor (GAO, 2001; DoD IG, 2009). Additionally, the GAO and DoD IG reports 
have expanded on the government’s lack of adequate market research relevant to 
determining the proper contracting strategy during the procurement planning of service 


contracts (GAO, 2002a; DoD IG, 2009). 


Requirements management is required to effectively define and meet the 
customers’ needs and expectations. Despite the importance of requirements management, 
the DoD IG and GAO have indicated poorly defined requirements and inadequate 
requirements management as a problem in service contracting (GAO, 2007b; DoD IG 
2009). Although the implementation of project management tools and processes such as 
cross functional teams is considered a best practice in service contracts and would 
improve the coordination and management of service acquisitions, the GAO has indicated 


the DoD lacks management structure and processes for managing service contracts 
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(GAO, 2002b, 2007b; DoD IG, 2009). The GAO has described the DoD’s current 
approach to service contract management as reactive and not fully addressing the key 
factors of success (GAO, 2007b). Oversight and surveillance are prudent to achieving 
contractors’ adequate performance of services and assist in precluding any contractor 
performance problems. Lack of oversight compromises the government’s ability to 
provide complete value to its constituents. Yet, DoD IG and GAO reports have 
consistently identified issues in service contract administration and oversight (GAO, 
2005, 2007a, 2007b; DoD IG, 2009). According to the GAO, the poor management of 
service contracts has undermined the government’s ability to obtain a good value for the 
money spent and contributed to the GAO’s decision to designate contract management a 


high-risk area for the DoD (GAO, 2001, 2007c, 2011). 


The DoD IG’s and GAO’s finding on deficiencies in the service contract process 


can be further explained by utilizing stakeholder theory. 


E. STAKEHOLDER THEORY 
Li; Definition and Implication for DoD Services Contracting 


Stakeholder theory can be used to analyze and discuss the similarities and 
differences that we anticipate will exist in defining and measuring the success of a service 
contract between the PM, PCO, and COR. In this section, we describe the foundations of 
stakeholder theory and how that theory relates to DoD service contracting. Using 
stakeholder theory can lead to a better understanding of both the conflicting and common 
interests the PM, PCO, and COR might have. This can help acquisition professionals to 
better understand why various stakeholders in DoD service contracting define and 


measure the success of service contracts differently. 


There have been numerous definitions of what a stakeholder is, but no definition 
is more important than that of Edward Freeman (1984). Considered a pioneer in the field 
of stakeholder theory, Freeman defined a stakeholder as any group or individual 
person(s) that can be affected by an organization or identity achieving its mission or 
objectives (Freeman, 1984). Freeman, Wicks, and Parmar (2004) of the Darden School 
of Business, have stated that stakeholder theory illustrates clear conflicts of interests 
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between the different stakeholders within a project. They stated that these conflicts of 
interests can have negative effects (such as inefficiencies and waste) on numerous 
projects. Differing goals and objectives of the various stakeholders drive these negative 
impacts (Freeman et al., 2004). Donaldson and Preston (1995) further explain that 
organizations which actively manage often conflicting interests of internal stakeholders 
fare far better in traditional measures of success, such as return on investment and profits, 
than those who do not (Donaldson & Preston, 1995). What this means for DoD services 
contracting is quite apparent. With the differing goals, objectives, and responsibilities of 
the various stakeholders (PM, PCO, COR) within DoD services contracting, it is prudent 
to assume that conflicts will arise with regard to how a service contract is processed, as 
well as answering the questions of how a successful service contract is defined and 
measured. While profit and return on investment are not applicable to defense services 
contracting, it is important to note the impact that managing stakeholder interests will 
have on the public sector, specifically in the area of public interest by reducing integrity, 
accountability, and transparency in the contracting process. By managing the 
stakeholders’ conflicting interests, the DoD may achieve some of the additional 


efficiencies received by its counterparts in the private sector. 


2. Stakeholder Management 


Stakeholder management is described as the management of the individuals and 
institutions that share a stake or interest in a project (Cleland, 1986). In the case of DoD 
services contracting, the “project” is the acquisition of the service. Cleland describes the 
principle justification for utilizing a stakeholder management perspective to be the need 
to recognize the ability of key stakeholders in influencing projects (Cleland, 1986). As 
presented earlier in this chapter, the PM, PCO, and COR all have different roles and 
responsibilities within DoD services contracting. As such, they routinely have conflicting 
interests and objectives of a service contract, which in turn may lead to conflicting 
definitions and measures of a successful service contract. Stakeholder theory holds that 
the various stakeholders will routinely have not just differing objectives and motives for 
outcomes within a project, but additionally will have differing ideas as to which factors 


are the most important for determining success of a project. This can present additional 
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future challenges when balancing the tradeoffs between the performances criteria (such 
as cost and schedule) of a contract (Project Management Institute, 2008). Understanding 
stakeholder theory and how these different stakeholders maintain conflicting interests that 
will impact the contract allows us to better prepare for mitigating the adverse results of 
such conflicting objectives. This relates directly to our research into the definitions and 
metrics for a successful service contract, as held by its chief stakeholders: PMs, PCOs, 
and CORs. Cleland states that positive stakeholder management can lead to cooperation 
within the project (or service contract) between the different stakeholders, resulting in 
enhanced project objective achievement. Lack of positive stakeholder management will 
result in the reduction in project objective achievement (Cleland, 1986). Translated into 
DoD services contracting, this means that proper managing and mitigating of conflicting 
stakeholder objectives will lead to more effective and efficient services contracts. 
Additionally, this may lead to a more standardized definition and measure of successful 
service contracts by DoD PMs, PCOs and CORs. Understanding this concept allows 
acquisition professionals to predict and even aid in control of the conflicting goals of 


different stakeholders within DoD services contracting. 


One approach to standardized measures would be S.M.A.R.T. metrics. 
S.M.A.R.T. is a tool utilized by corporate officers, managers, and supervisors in helping 
to determine quantifiable metrics and objectives for their project and company mission. 
The establishment of objectives and the development of action plans are extremely 
important steps in any organizations management process (Doran, 1981). Each metric 


should have the following attributes: 


e Specific: It targets a specific area for improvement 

e Measurable: It quantifies or provides an indicator of progress. 

e Assignable: Someone is determined to have responsibility for it. 

e Realistic: It can realistically be obtained, given resource constraints. 

° Time related: It is specific to when the results can be achieved. (Doran, 
1981) 
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F. SUMMARY 


In this chapter, we introduced past and current literature on the service contracting 
process, the acquisition team, roles and responsibilities of service acquisition personnel, 
goals and objectives of service acquisition personnel, deficiencies in service contracts, 


and stakeholder theory. 


The next chapter describes our research methodology, including the type of 


analysis, the location of data collection, and the interview questions. 
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Hl. RESEARCH METHODOLOGY 


A. INTRODUCTION 


In this chapter, we provide an overview of how we collected and analyzed our 
data in order to achieve our objectives and answer the research questions discussed in 
Chapter I. We discuss the formulation of the interview questions, the survey, and our 
analytical process. In this chapter, we also include a description of the qualitative 
methods we used in analyzing the data collected from the Navy contracting activities. 
Our purpose was to conduct an exploratory research analysis of the definition and 
measurement of successful services contracts. The objective of the research was to build 
upon the understanding developed in prior research and explain what metrics are utilized 
to help identify the factors that influence the efficiency and effectiveness of service 
contracts. We analyzed the collected data qualitatively in order to draw conclusions 
about the definition and measurement of success of service contracts, as well as the 


commonalities and distinctions among the stakeholders. 


B. DATA COLLECTION 


We conducted interviews and surveys with the PMs, PCOs, and CORs located at 
and associated with the following contracting commands: Naval Sea Systems Command 
(NAVSEA), Naval Air Systems Command (NAVAIR), and Naval Fleet Logistics Center 
(FLC) Philadelphia. Table 1 identifies the number of stakeholders interviewed and 


surveyed. 
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Table 1. Interviews and Surveys Conducted 

















TOTAL 
STAKEHOLDER # # RESPONDENTS 
INTERVIEWED | SURVEYED 

CONTRACTOR 2 N/A 2 
PRINCIPAL CONTRACTING OFFICER/ 
CONTRACT SPECIALISTS 22 N/ A 22 
CONTRACTING OFFICER REPRESENTATIVE 13 1 14 
PROGRAM MANAGER 1 2 3 

















iM Participating Commands 


NAVSEA’s overarching mission is to engineer, build, buy, and maintain ships, 
submarines, and combat systems to meet the United States Navy’s current and future 
operational needs. NAVSEA’s fiscal budget compromises 25% of the Navy’s entire 
budget, at nearly $30 billion. NAVSEA manages over 150 acquisition programs, to 
include foreign and domestic military sales. NAVSEA is an essential system command 
for the United States Navy, providing efficient resources and support for the nation 


(NAVSEA, 2012). 


NAVAIR’s mission is to provide full life cycle support of naval aviation aircraft, 
weapons, and systems for the United States Navy; to provide the right capability, at the 
right time, at the right cost. This support includes research, design, development, systems 
engineering, acquisition, test and evaluation, training, repair, and logistics support. They 
further provide support to their program executive officers with their assigned duties of 
meeting cost, schedule, and performance requirements within their respective programs 


(NAVAIR, 2012). 


FLC Philadelphia is a subordinate command of Naval Supply Systems Command. 
FLC Philadelphia’s mission is to deliver sustained global logistics resources to the United 
States Navy and the joint warfighter. FLC Philadelphia manages supply chains that 
provide material for Navy aircraft, surface ships, submarines, and their weapon systems. 
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Additionally, they provide logistics support services, material management and 
warehousing services, contracting and acquisition, as well as all food service operational 


support ashore and afloat (Naval Supply Systems Command, 2012). 


2 Interview Questions 


We developed interview questions and a survey to answer the following core 
research questions: 
e What is the definition of a successful service contract? 


° How is the success of a service contract being measured? 


The survey and interview questions consisted of the following five questions: 
1. What is your branch of service or service affiliation? 

2s What is your current functional role? 

2: What is your DAWIA level certification? 


4. How do you define a successful service contract and what factors are 


included in your definition? 


5. How do you measure the success of a service contract and what metrics 


are included in your measurements? 


The first two questions of the survey and interviews identify the demographics of 
the respondents to establish broad categories. The third question, on DAWIA level 
certification, gives insight into the respondents’ level of training, experience, and 
education. The DAWIA establishes a procedure through which acquisition workforce 
personnel are recognized as having achieved qualification in their core discipline. 
Certification is the procedure through which a DoD component determines that an 
employee has met the education, training, and experience standards. These standards are 
required for a career in government acquisition, technology, and logistics fields. DAWIA 
level certifications are categorized as Level I, Level II, and Level II]. The remaining two 
questions address the primary purpose of our research, to determine how stakeholders 


define and measure the success of service contracts. 
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C. ANALYTICAL PROCESS 


We chose open-ended questions to gain qualitative data for our research and to 
accommodate differentiated responses from varied stakeholders. In Chapter IV, we 
present this qualitative data in graphical and tabular formats. By consolidating the data 
into categories, we were able to conduct further analysis. We identify commonalities and 
relationships in the data to determine the answers to our research questions, identified in 
Chapter I. Additionally, we determine if the stakeholders (PM, PCO, and COR) are in 
fact utilizing effective and quantifiable S.M.A.R.T. 


D. SUMMARY 


In Chapter II, we identified which Navy organizations were surveyed and 
interviewed, the formulation of our survey and interview questions, and how the data was 
collected and analyzed. In Chapter IV, we present the data, analyze of the findings, and 


make recommendations. 
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IV. INTERVIEW INSTRUMENT, RESULTS AND ANALYSIS 


A. INTRODUCTION 


In this chapter, we examine the responses to the questions answered in our 
interviews. The objective of this research was the collection of data through the use of 
interviews and surveys in order to define and measure the success of service contracts. 
We utilized a standard script of five questions presented to each stakeholder. The 
interviews were conducted at the following contracting activities: NAVSEA, NAVAIR, 
and FLC Philadelphia. Additionally, interviews were conducted with CORs associated 


with these contracting activities. 


B. INTERVIEW QUESTIONS 


The interview consisted of three demographic questions and two core research 


questions: 
1. What is your branch of service or service affiliation? 
2. What is your current functional role? 


3. What is your DAWIA level certification? 


4. How do you define a successful service contract and what factors are 
included in your definition? 


5. How do you measure the success of a service contract and what metrics 
are included in your measurements? 


The focus of the demographic questions was twofold: to establish the individual’s 
level of knowledge, experience, and education (DAIWA level certification) and to 
identify the individual’s role (PM, PCO, or COR) and branch of service (Navy, Marine 
Corps, Air Force, or Army). 


The focus of the core research questions was to determine the key definitions and 
factors determining the definition of a successful service contract and to determine the 


metrics used to measure a successful service contract. 
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C. PROJECT MANAGER FINDINGS 


In the course of our research, we were only able to survey and interview three 
PMs. We found that service contracts generally do not have a PM designated and that the 
role is typically filled by the incumbent COR. The definitions and measurements of 
success given by the proxy PMs aligned with the responsibilities of a traditional PM 
focusing on cost, schedule, and performance. We found that the PMs factored clarity in 
the statement of work, effectiveness, unproblematic, schedule and tracking costs as key to 
defining the success of a service contract. Further, the PMs stated that end user 
evaluations, customer satisfaction, and performance (meeting the requirements in the 


statement of work) were essential in measuring the success of a service contract. 


D. DATA ANALYSIS ON PRINCIPAL CONTRACTING OFFICER 
1. Overview of Data Collected on Principal Contracting Officer 


Of the 22 interviews conducted, 16 were with PCOs and six were with contract 
specialists. The DAWIA level certification of the PCO and contract specialists ranged 
widely: 15 had a level III certification, four had a level II certification, and three had a 
level I certification. Based on the interviews of the PCOs and contract specialists, the 
responses for defining the success of service contracts were classified into nine 


categories, discussed in Table 2. 


Table 2. PCO Success Definitions 








SUCCESS COMMON DESCRIPTORS 
CATEGORY 
Schedule e Contract is executed on time 
(Outcome) e Doing it in a timely manner 


e Meeting milestones 
e Getting contracts awarded on time 


e Keep to the schedule 








Unproblematic e “Don’t have to do a lot of administrative modifications and 
adjustments to the end contract” 





(Process) 
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“Successful bid process where there is not any animosity 
between the parties.” 


Relatively unproblematic 
Runs smoothly 
Operates seamlessly 


“Not hearing from the customer.” 





Customer 
Satisfaction 


(Outcome) 


Meeting customer requirements 

Business like concern for the customer 
Motivated contractor performance 

Got what we paid for 

Satisfies the mission requirements 

Delivers the service that your customer expected 


“Responsive to the needs of the contract and the needs of 
the activity.” 


Customers are happy. 





Well-Defined 


Statement of work is clearly defined 








Requirements : 
: Well-defined requirement, 
(Process) seeces ; : < ‘ 
Satisfies the test requirements identified in the 
Performance Work Statement.” 
“How are the services defined and is it defined in a way that 
it is clear to industry.” 
“Understanding needs of the customers.” 
Communication Customer involvement 
(Process) Partnership with industry 





“Contractor has to be motivated to perform.” 


“Contracting group as well as the programs side all know 
what needs to get done.” 


“Able to work with all parties involved.” 


“A lot of integration up front with the customer and our 
office.” 


“Mutual understanding as to what is required, the manner in 


which it will be furnished and then how we will monitor 
those services or the performance, a good relationship.” 
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“You are able to talk and work things through and come up 
with solutions.” 








Contract Vehicle “Choosing the right contract.” 
(Process) “Flexibility in your contract vehicle.” 
Efficient “There would be some sort of result in efficiency that came 
(Process) about during the performance of that particular contract.” 


Adherence to 
Regulation 


(Process) 


“Maintain rules and regulations.” 








Maintain Costs 


(Outcome) 





“Services at the right price.” 
“Spending dollars wisely.” 

“Keep within cost.” 

No overruns 

“Contractor’s costs well controlled.” 
“Perform the work within the budget.” 


“Not running into issues where the contractor’s been 
performing but hasn’t had funding.” 





Of the 22 contracting officers and specialists interviewed, 68% of respondents 


specified customer satisfaction and maintaining costs as a definition of success. Fifty-five 


percent of respondents indicated unproblematic as equivalent definitions of the success of 


a service contract. At 45%, the third most prominent definition of success was well- 
p 


defined requirements. 


Thirty-six percent of the respondents indicated communication 


relevant to the success of a service contract. Twenty-three percent of the contracting 


officers indicated maintaining schedule relevant to the success of a service contract. The 


final three criteria—adherence to regulations, contract vehicle, the efficiency—were 


specified by 5%, 9%, and 9% of the respondents, respectively. PCO definitions of 


success response rates can be seen in Figure 4. 
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Figure 4. PCO Success Definitions 


2 Overview of Data Collected on Contracting Officers Metrics 


Based on the interviews of the contracting officer and specialist, the responses for 
measuring the success of service contracts were classified into five general categories, 


depicted in Table 3. 


Table 3. PCO Measures of Success 











METRIC COMMON DESCRIPTORS 
CATEGORY 
Past Performance e Contractor Performance Assessment Reporting System 
(Outcome) (CPARS) 
e Past Performance Information Retrieval System 
(PPIRS) 
e COR annual report, mini CPARS, COR functional 
reviews 





e COR verbal periodic review 
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COR review, customer reviews 
Prior contract performance 
Subjective 


Past history, or requests from customer to write a 
contactor a letter or amend the contract 


Feedback from customers on contractor performance 
Evaluation and recording 
Ability to get future contracts 


A way to track contractors 





Surveillance Plan 


(Process) 


Is the contractor performing in accordance with the 
Quality Assurance Surveillance Plan (QASP) 


Built in measurements that measure quality 
Deficiency reports 

Measurements of successful performance 
Realistic expectations 


“Performance evaluation factors to gather from the 
contract” 


“Frequency of when the service is performed” 
“Something to easily put your eyes on” 
“Performance standard metrics” 

Quality control 

“Requirement summary of the various tasks” 
“How well the contractor is performing” 


Monitoring turnaround times 








Track Costs 


(Outcome) 





“Burning through ceiling faster than anticipated” 


“Tracking: spend rates, proposed man hours compared 
to what is being delivered.” 


“Measuring performance to cost controls.” 
Measuring workload status 
Maintaining good cost control 


“Being involved with my COR early to say, are you 
looking at your burn rate?” 
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“Performance is measured via cost control tracking 
hours, dollars, and parameters to hours and dollars, not 
over or under running.” 


“Making sure that the costs in the end are at the 
proposed level and within budget, tracking different 
modifications to obligations, milestones on how long 
things are taking.” 





Customer Satisfaction 


(Outcome) 


Customer feedback 


“customer service surveys given to the activity for the 
contracting office, immediate feedback on vendor 
issues, am I (the agency) comfortable (uncomfortable) 
with the guy (the contractor).” 


“Am I (the agency) not comfortable with the guy (the 
contractor)” 


“A lot of times that’s (customer satisfaction) is built 
around personal interaction with the contractor more so 
than actual good or bad performance.” 


“Level of angst, complaints from discontent customers.” 








Track Schedule 


(Outcome) 





Meeting milestones 

Awarded on time 

Not a break in service 

“Execution of procurement planning agreement” 


“Internal work in progress (WIP) reports to track the 
number of days the contract has been in house or 
workload assigned in the system with no errors, timely 
delivery, all deliverables received, on schedule, are the 
deliverables on time.” 
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Of the 22 PCOs and contract specialists interviewed, 59% of the respondents 
listed past performance as a measurement for service contract success. Having a 
surveillance plan was also indicated as a measurement for service contract success by 
59% of respondents. Forty-one percent of the respondents identified customer 
satisfaction, 27% identified tracking costs, and 23% identified tracking schedule. PCO 


measures of success response rates are depicted in Figure 5. 
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Figure 5. PCO Measures of Success 


3. Research Findings from PCO Interviews 


Findings for the PCO produced no decisive definition of success; no category 
received an absolute majority of responses. Several categories did demonstrate a strong 
predilection, with over half of the respondents identifying the categories as indicators of a 
successful service contract. The most frequent definitions of a successful service contract 
were satisfying the customer and maintaining costs, both of which had 15 responses. Of 
nearly equal significance was the category unproblematic with 12 responses. Well- 


defined requirements were indicated as a definition of success by 10 respondents. The 
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Department of Defense COR Handbook specifies a properly written Statement of 
Objectives (SOO) or Statement of Work (SOW) as increasing the likelihood of success 
(Director, Defense Procurement and Acquisition Policy, 2012). Eight PCOs identified 
communication expressed between primarily the contractor, customer, and PCO as an 
indicator of success. Communication is a trait necessary within a business organization; it 
allows the acquisition team to work together to successfully purchase a service (Garrett, 
2010). In comparison with the FAR’s performance requirements of cost, quality, and 
timeliness of the delivered service, five of the respondents indicated schedule, and as 
previously mentioned 15 indicated maintaining costs as a definition of success. 
Categories deemed to be defining factors of success to a lesser extent were contract 
vehicle, efficiency, and adherence to regulation with two, two, and one responses, 


respectively. 


Similar to contract success definitions, the findings yielded no definitive 
measurement of success. Each response was grouped into five categories. With 
13 responses each, the two most frequent responses were past performance and 
surveillance plan. Customer satisfaction yielded nine responses; tracking costs received 
six, and tracking schedule five. PCOs’ responses revealed the measurements of service 
contract success are in a majority of cases very subjective and ambiguous. As summed 
up by several respondents: “At the end of the day, we don’t have a numeric score of yes, 
this contractor gets a B+” and “[we have] CPARS ratings—but a lot of them are pretty 
subjective.” An analysis of Figure 5 reveals, with respect to outcome-based 
measurements, past performance as documented through the COR annual reports in 


CPARS is the most common approach to measuring contract success. 


4. Outcome Versus Process 


Our findings on PCO definitions and metrics of a successful service contract 
demonstrate both an outcome- and process-oriented approach. The PCOs’ definitions of 
a successful service contract lined themselves more to a hybrid-oriented approach, with 
3 of the 9 categories of responses being outcome oriented and 5 of 9 categories process 


oriented. The two most common response categories were customer satisfaction and 
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maintained costs: both are outcome-based definitions. The findings illustrate that the 
definition of a successful service contract incorporates both outcome- and process-driven 
criteria. While the two prominent definitions of success were outcome based, the majority 
of the definitions were process based. The majority of process-based definitions could be 
a result of the PCOs’ greater involvement in the pre-award phase versus the post-award 
phase of the service contract process. The contracting offer is able to determine, through 


his pre-award actions, the subsequent results with respect to a successful service contract. 


The results on PCO measurements of success show that four out of five response 
categories for the measurement of contract success were outcome-oriented approaches. 
The two significant categories, past performance and surveillance plan, were process and 
outcome based, respectively. This mix of process and outcome are linked. The 
surveillance plan, a process approach, facilitates the measurement of past performance, 


an outcome approach, in the form of CPARS. 


Of the five measurements, we found past performance, track costs, and track 
schedule met all the criteria of the S.M.A.R.T. tool. As previously discussed, the 
S.M.A.R.T. tool is utilized to assist corporate managers and supervisors to determine 
quantifiable metrics and objectives. Although surveillance plan accounted for 59% of the 
responses, it failed to properly address the criteria of realistic in S.M.A.R.T. Therefore 
the most quantifiable and appropriate measures for success of a service contract should 


be past performance, track costs, and track schedule. 


Table 4. S.M.A.R.T. PCO Metrics 




















CATEGORY S M A T 
Past Performance V V V J Nl 
Surveillance Plan V V V J 
Track Costs V V V J a 
Customer Satisfaction ] 

Track Schedule V V J J A 
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E. DATA ANALYSIS ON CONTRACTING OFFICER REPRESENTATIVE 
L., Overview of Data Collected on Contractive Officer Representative 


Fourteen CORs were interviewed for the purposes of this research. DAWIA level 
certification was obtained by 35% of CORs interviewed. This represented a smaller 
portion than we originally anticipated. However, DAWIA certification is not a 


requirement for appointment to a COR position. 


Based on the COR interviews, we arranged the responses for defining the success 
of service contracts into four categories (annotated as a process or outcome) with 


common descriptions of each category, depicted in Table 5. 
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Table 5. COR Definitions of Success 





SUCCESS COMMON DESCRIPTORS 
CATEGORY 
Schedule e Timeliness, (on) schedule 
(Outcome) e Deliverables met on time. 
Performance e Responsive to statement of work (SOW) 
(Outcome) e Statement of objectives (SOO) 


e Satisfactory deliverables 
e Performance satisfactory with terms and conditions of contract 
e Maintained performance within conditions of contract 


e Objectives meet requirements. 














Well-Defined e No deviation in work performed 
Requi Bak 
Seopes <dunen e Clarity in SOW/SOO 
ents 
(Process) e Requirements met needs of the end user 
e Clearly written contract 
e Well-defined criteria that are documented 
e Clarity of SOW allows for proper estimating in terms of 
preparing government cost estimation 
e Specific SOW requirements which mitigate scope creep by 
PM/PCO/COR. 
Communication e Continuous feedback by and between COR/contractor/end user 
(Process) e Clear language and communication between 
contractor/customer 
e Open communication 
Maintain Costs e Adherence to budget, no cost overruns 
(Outcome) e No project creep 


e Fair and reasonable 

e Cost in line with services required 
e Fair cost to government 

e Meets price guidelines 


e Deliverables on time and within budget 
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Of the 14 CORs interviewed, 71% responded that well-defined scope and 
requirements were key factors in defining a successful service contract, while 58% of 
CORs responded that schedule was a key factor in defining the success of a service 
contract. Performance had the third highest response rate at 50%. The categories of 
communication and maintaining cost represented 21% of the responses. The definitions 


of success response rates are depicted in Figure 6. 


co, COR Definitions of Success (n=14) 
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Figure 6. COR Definitions of Success 


2 Overview of Data Collected on Contracting Officer Representative 
Metrics 


Based on the COR interviews, we categorized the responses for measuring the 
success of service contracts into seven categories (annotated as a process or outcome) 


with common descriptions of each category, depicted in Table 6. 
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METRIC 
CATEGORY 
Performance 


(Outcome) 


Table 6. 


COR Measures of Success 


COMMON DESCRIPTORS 


Contract performed in accordance with terms and 
conditions of the contract 


Deliverables met quality requirements in SOW 


Proven response times on service contracts 





End User Evaluation 


Final product/service met needs as of end user as 
evaluated by end user 

















(Outcome) 
e Summary reviews by technical experts 
Track Schedule e On time 
(Outcome) e Timeliness 
e All deliverables received 
Communication e Feedback loop maintained between contractor 
(Process) e COR, PM, and/or end user 
e Required reporting delivered/received 
No Rework e No loss time on rework 
(Outcome) e No additional resources utilized for corrections to end 
products/services 
Track Costs e Final costs remained within projected costs 
(Outcome) e No cost overruns 
e Billable hours matched contractual levels 
e Effectively tracked cost and relate costs tracking to 
customer 
No Protest e No contractor protests 
(Outcome) e Unproblematic source selection and award process 
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Of the 14 CORs interviewed, 71% responded that performance was the key factor 
in defining the success of a service contract. Track schedule accounted for 28% of 
responses. End user evaluation represented 21%. Adherence to budget represented 14% 
of responses. Each of the following criteria respectively accounted for 7% of responses: 
good communication, no protest, and no rework. COR measures of success response 


rates on depicted in Figure 7. 
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Figure 7. COR Measures of Success 


3. Research Findings from COR Interviews 


The research findings from our interviews conducted with CORs clearly 
demonstrated varying definitions for a successful service contract, as well as numerous 
ways in which to measure this success or lack thereof. Of the 14 CORs interviewed, the 
highest response category was performance, consisting of 10 responses; followed by the 


category of track schedule, with four responses. The remaining five categories of 
4] 





responses yielded as high as three responses or as few as one. Some of the variability 
found in the form of measuring service contracts can be explained by the numerous types 
of services being contracted. These CORs represented contracts that range anywhere 
from medical supplies to legal counsel. The heavily diverse nature of these service 


contracts may have contributed to the lack of a real consistent form of measurement. 


The definitions of successful service contracts was less variable in comparison to 
the responses received for measuring success of service contracts. Of the 14 CORs 
interviewed, 10 responded that well-defined scope/requirements was the most important 
criteria for defining a successful service contract, while the categories of schedule and 
performance each had eight and seven responses, respectively. Communication and 
maintaining costs each had three responses each. This clearly shows that within the COR 
stakeholder group, there exists a more standardized definition of what is a successful 


service contract. 


4. Outcome versus Process 


Our findings on COR definitions and metrics of a successful service contract 
demonstrate an outcome-, process-, or hybrid-oriented approach. The CORs’ definition 
of a successful service contract lined itself more to a hybrid-oriented approach, with three 
of the five categories of responses being outcome oriented and two of the five categories 
being process oriented. The two most common response categories were well-defined 
scope/requirements (process oriented) and schedule (outcome oriented). The significance 
of this finding is that it clarifies the importance of both outcome- and process-driven 
criteria held by CORs in determining the key factors defining a successful service 


contract. 


The CORs’ metrics for a successful service contract are overwhelmingly outcome 
driven, with six of the seven response categories being outcome oriented. Additionally, 
the two most prevalent responses were performance (outcome) and track schedule 
(outcome). These findings clearly demonstrate the importance that outcome-specific 


measurements have in the success of a service contract as determined by CORs. 
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When analyzing the findings on metrics for a successful service contract, we 
focused on whether each category was found responsive to the demands found in the 


S.M.A.R.T. objectives protocol, as presented in chapter II 


Of the seven measurements, we found performance, track schedule, and track 
costs met all the criteria of the S.M.A.R.T. tool. As previously discussed, the S.M.A.R.T. 
tool is utilized to assist corporate managers and supervisors to determine quantifiable 
metrics and objectives. Although end user evaluation accounted for 21% of the 
responses, it only met the criteria of assignable in S.M.A.R.T. Therefore the most 
quantifiable and appropriate measures for success of a service contract should be 


performance, track schedule, and track costs. 

















Table 7. Success Measurement Categories and S.M.A.R.T. 

CATEGORY S M A R T 
|Performance | V | | V | vf Vv 4 

End User Evaluation V 

Track Schedule V V V V V 
|Communication [| | | V J] [| 4] 

No Rework V V 

Track Costs V V V V V 
|NoProtest | V | vV | V J Jf 4] 


























We discovered that the majority of the categories for COR measurements of a 
successful service contract failed to properly address the objectives of S.M.A.R.T. This 
clearly demonstrates that fact that currently a substantial portion of CORs are not 


properly measuring success of service contracts. 
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F. CONTRACTOR FINDINGS 


Despite the emphasis of this research on the PCO and COR, we were able to 
interview 2 contractors and gain their perspective on the definition and measurements of 
success in a service contract. The contractors’ definition of success includes winning 
repeat business, finding ways to innovate, satisfying the customer, helping the customer 


shape the SOW, and helping the government “folks get up to speed.” 


The contractors’ measurement of success is profit and the ability to perform and 
execute the specific task. Based on this response, this measurement was aligned with 
FAR 46.105: “The contractor is responsible for carrying out its obligations under the 
contract” (FAR, 2012). Success is also measured in terms of spend rates—that is, over or 
under budget (how many people are put to work and how many people have to be hired 
to complete the task). The definitions and measurements of success allude to maximizing 


profit as the overarching goal of a contractor. 


G. CONTRACITNG OFFICER REPRESENTATIVE AND PRINCIPAL 
CONTRACTING OFFICER COMPARISON 


1B Contracting Officer Representative and Principal Contracting Officer 
Similarities in Definitions of Success 


Our research revealed the following similarities between CORs and PCOs when 


defining the success of a service contract: 


e Schedule 

e Maintaining costs 

e Communication 

° Well-defined scope and requirements. 
a. Schedule 


The data shows both the PCO and COR identify maintaining schedule as a 
common definition of success. PCOs and CORs are both members of the acquisition 


team guided by the objectives and regulations of the FAR. In service contracts, the COR 
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often performs the functions of a PM; thus, schedule is an important factor to the COR as 
well as progressing the contract through the acquisition life cycle. Additionally, the FAR 
states the acquisition team must satisfy the customer in terms of cost, quality and 
timeliness of the delivered goods or services (FAR, 2012). Furthermore, the COR 
handbook states that the COR should ensure that the contract for goods or services is 
timely and highlights schedule as a key assessment factor (Director, Defense 
Procurement and Acquisition Policy, 2012). The schedule is the most visible and 
scrutinized performance measure for both the PCO and the COR. For example, the 
workload of the PCO is driven by their WIP report, which tracks the number of days the 
contract has been assigned in the system. One of the key assessment factors of the COR 


is schedule (Director, Defense Procurement and Acquisition Policy, 2012). 


b. Maintain Costs 


Sixty-eight percent of PCOs and 21% of CORs defined maintaining costs 
as a definition of success. Given the budget constraints, DoD acquisitions are limited by 
scarce resources. As such, maintaining costs is a high priority. Failure to maintain 
budget could result in a possible violation of the Anti-Deficiency Act, defined as an 
obligation in excess of available funds. As mentioned earlier, the COR often assumes the 
responsibilities of the PM on service contracts; therefore, cost becomes a responsibility. 
The data shows a difference of 47% between PCO and COR responses. The disparity is a 
result of the PCOs’ accountability for contract administration; therefore, the PCO places 
a higher emphasis on maintaining costs. This is supported by the FAR (2012), which 
states that PCOs “shall ensure that the requirements of 1.602-1(b) have been met and that 


sufficient funds are available for obligation.” 


c. Communication 


The data concludes that CORs and PCOs share communication as an 
attribute definitive to success. Common to both, the FAR states the federal acquisition 
system will foster cooperative relationships between the government and its contractors. 
More importantly in each case, successful communication and continuous feedback 
between PCO and COR and acquisition team members contributes to a service contract 
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success. When all members of the project team communicate properly, clear and concise 
objectives are conveyed to the team and the goal is understood and _ identified. 
Additionally, clear communication is fundamental to the resolution of deficiencies in 


service contracts. 


d. Well-defined Scope and Requirements 


The data shows 71% of the CORs versus 45% of the PCOs define success 
as a well-defined scope and requirements. The COR and the PCO are focused on 
meeting the activity’s need and satisfying all technical aspects of the SOW. A well- 
defined scope and set of requirements lay the foundation the required services. 
Therefore, the PCO and the COR have a vested interest in having a well-defined scope 
and meeting requirements in the SOW. With a proper SOW, contract performance is 
more likely to be successful. One of the key pre-award duties in which the COR may be 
involved is documenting requirements. The requirements package is critical to the 
success of an acquisition because it commits the funds and establishes the basis for a 


contractual action. 


2. Contracting Officer Representative and Principal Contracting Officer 
Differences in Definitions of Success 


Our research revealed the following differences between CORs and PCOs when 


defining the success of a service contract: 


e unproblematic, 

e customer satisfaction, 

e adherence to rules and regulations, 
e contract vehicle, and 

° efficiency. 
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a. Unproblematic 


Fifty-five percent of the PCOs stated unproblematic as a definition of 
success. The PCO is responsible for administrating all aspects of the contract; therefore, 
a problem-free contract reduces the workload of additional modifications and contract- 
related issues. Increases in workload are even more substantial given the inadequate size 
of the acquisition workforce. Contrarily, CORs are less concerned with contract 
administration and are more focused on technical evaluation factors. The COR is more 
concerned with the surveillance plan and delivering problems to the contractor. Problems 
are not a true concern to the COR. According to the COR handbook, the COR is 


responsible for bringing any issues or performance problems to the attention of the PCO. 


b. Customer Satisfaction 


Sixty-eight percent of the PCOs conveyed customer satisfaction as a 
definition of success, demonstrating the emphasis PCOs place on customer satisfaction. 
The PCO is more customer service oriented in comparison with the COR, whose primary 
purpose is to monitor the performance of the contract. A PCO must evaluate all relevant 
surrounding circumstances to discern the customers’ expectations (Hirsch, 1986). 
Additionally, customer satisfaction plays a central role in determining contractors’ past 
performance, an evaluation factor required by the FAR that results in contract renewal or 


follow-on contracts. 


c. Adherence to Rules and Regulations 


Five percent of the PCOs stated adherence to rules and regulations was a 
definition of success. In accordance with the FAR the PCO ensures all requirements of 
law, executive orders, regulations, and all other applicable procedures, including 
clearances and approvals, are adhered to in government contracting. The COR, while a 
representative of the PCO, has no true authority to affect change such as alter the terms 
and conditions of the contract; therefore, the COR places no emphasis on compliance to 
rules and regulations. The COR’s only liability is in the form of unauthorized acts. 
Adherence to rules and regulations is a core function and inherent to the position of PCO; 


therefore, adherence to rules and regulations is implied. 
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d. Contract Vehicle 


Nine percent of the PCOs specified the contract vehicle as definitive of 
success. The CORs gave no indication to contract vehicle as a definition of success. This 
is owing to the possible lack of integration of the COR in the pre-award phase of service 
contract process. The Department of Defense COR Handbook (Director, Defense 
Procurement and Acquisition Policy, 2012) only lists pre-award activities the COR may 


be involved with and makes no reference to required involvement. 


e. Efficiency 


Nine percent of PCOs interviewed specified efficiency as a definition of 
success. The CORs made no acknowledgment of efficiency as a definition of service 
contract success. As previously determined, the CORs’ definition of success is primarily 
outcome oriented. The COR is concerned with tracking the contract’s compliance 
according to its terms and conditions and delivery of the final product. In contrast, the 
PCO definition of success is primarily process driven and processes lend themselves to 
continuing improvement. As a PCO explained, contract efficiency leads to a reduction in 
contract risk, allowing the government to switch contract vehicles from a risk-burdened 


cost contract to a fixed contract, which transfers risk to the contractor. 


3. Contracting Officer Representative and Principal Contracting Officer 
Similarities in Measurements of Success 


Our research revealed the following similarities between CORs and PCOs when 


measuring the success of a service contract: 


e past performance and performance, 

° track costs, 

e track schedule, and 

e customer satisfaction and end user evaluation 
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a. Past Performance and Performance 


Data shows PCOs and CORs measure past performance and performance 
59% and 71%, respectively. _PCOs’ and CORs’ measures of performance are both 
outcome-based measurements. Both measures employ user feedback as a gauge of 
success. The COR measures performance according to the surveillance plan, which 
provides input to the PCO’s measurement of past performance. Based on the research, 
performance and past performance are two of the most relevant tools used for measuring 


sucCeSS. 


b. Track Costs 


Data revealed PCOs and CORs measure tracking costs and adherence to 
budget 27% and 14%, respectively. This relates directly to the responsibilities of both the 
PCO and the COR. As outlined in the COR handbook, the COR must conduct 
surveillance to maintain costs (Director, Defense Procurement and Acquisition Policy, 
2012). The FAR specifically mandates that PCOs ensure adhering to procurement laws 
and regulations such as the Anti-Deficiency Act (Federal Acquisition Regulation, 2012). 


c. Track Schedule 


We found that the CORs and the PCOs placed similar emphasis on 
tracking schedule, with response rates of 28% and 23%, respectively. We found that the 
response rates for tracking sechdule were relatively low given the the requirements of 
both the COR and PCO, in accordance with the COR handbook, to ensure that contract 
performance is timely and within scope of the work (Director, Defense Procurement and 
Acquisition Policy, 2012). Additionally, the FAR states that the COR and PCO must 


satisfy the customer in terms of timeliness of the delivered service (FAR, 2012). 


d. Customer Satisfaction and COR End User Evaluation 


Data revealed the PCOs placed a higher importance on customer 
satisfaction than the CORs; the two groups had response rates of 41% and 21%, 
respectively. This demonstrates that the PCO places more of an emphasis on customer 


satisfaction as a measure of success than the COR. The low response rate from CORs 
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was surprising given the close interaction of CORs with end users and technical 
evaluators within services contracts. However, CORs are typically nominated for their 
technical expertise and not their customer relationship skills. Conversely, PCOs’ 
positions are the customer service oriented. Additionally, in certain cases the COR is 


also the customer. 


4. Contracting Officer Representative and Principal Contracting Officer 
Differences in Measurements of Success 


Our research revealed the following differences between CORs and PCOs when 


measuring the success of a service contract: 


° no rework, no protest, and good communication; and 
e PCO surveillance plan 
a. No Rework, No Protest, and Good Communication 


CORs listed no rework, no protests, and good communication as measures 
of a successful service contract. However, it is prudent to note that the response rate for 
all three categories was only 7% each, equating to only one responder in each category. 
As such, these categories represent a less than significant metric for success in service 


contracts. 


b. PCO Surveillance Plan 


The PCO’s surveillance plan represented one of the highest response rates 
at 59% (13 respondents out of 22). The FAR states that government contract quality 
assurance shall be performed at such times and places as may be necessary to determine 
that the services conform to contract requirements (FAR, 2012). The COR handbook 
clearly states that the QASP is an important tool for assessing the service contract for the 
COR (Director, Defense Procurement and Acquisition Policy, 2012). However, the 
CORs exhibited zero responses for surveillance plan, demonstrating a possible lack of 
training or that CORs’ experience involves contracts below the simplified threshold. 
Further, the QASP was prepared by the contractor. According to the FAR, “The 


Government may either prepare the quality assurance surveillance plan or require the 
50 


offerors to submit a proposed quality assurance surveillance plan for the Government’s 


consideration in development of the Government’s plan” (FAR, 2012). 


H. CONCLUSIONS 


Our findings corroborate the deficiency found within the GAO report on best 
practices in acquisition services, that there are few services contracting—related annual 


performance metrics (GAO, 2002). 


Further, we found that there is no standardized definition or measurements for 
success of service contracts. While similarities do exist between the definitions and 
measures of success of a service contract, the levels of emphasis placed on those similar 


categories in many cases were disproportionate. 


In addition, those metrics in place were found to lack both in terms of their ability 


to be quantified and their ability to meet the requirements of S.M.A.R.T. measurements. 


We discovered that the differing objectives and duties of the stakeholders clearly 
affected the factors that each stakeholder emphasized when defining and measuring the 
success of service contracts. Stakeholder theory identifies the conflicts that arise between 


the acquisition team and how these conflicts can lead to this lack of uniformity. 


In our research, we identified a clear lack of establishment of PMs within DoD 
services contracting. In many cases, the incumbent COR was the PM. This supports 
GAO findings that the DoD lacks the proper management structure and processes for the 
managing services contracts (GAO, 2002b, 2007b; DoD IG, 2009) 


Finally, we discovered that no uniform certification process was established and 
required for CORs within services contracting at the time of our research. This validates 
the GAO report’s conclusions on defense acquisition workforce training, stating that a 
lack of training for defense acquisition workforce personnel continues to plague DoD 
services contracting efforts (GAO, 2002). To our knowledge, reform measures are in 


process for the COR training certification process. 


We discovered the COR is not as involved in the pre-award phase of service 


contracts. According to the COR handbook, the COR is responsible for preparing the 
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SOW/PWS and surveillance plan. Our COR data indicates no reference to surveillance 
as a measure of success. Typically, the person who develops the SOW/PWS also 
develops the surveillance methods. The CORs’ lack of reference to a surveillance plan 


implies no significant involvement in developing the SOW/PWS. 


We found that the majority of PCOs put a large emphasis on past performance 
and surveillance plans, yet QASP is only mandatory for use in acquisitions in excess of 
simplified acquisition threshold. Additionally, CPARS is only mandated for services 
acquisitions in excess of $1 million. Yet, over 83% of United States federal acquisitions 
have a dollar value on average of $25,000. Therefore, the majority of acquisitions are not 
required to possess any surveillance plan, nor do they require any evaluation on past 
performance. Only 1% of federal acquisition contracts are over $1 million (Garrett, 


2011). 


I. RECOMMENDATIONS 


A standardized definition of a successful service contract must be implemented, 
incorporating the proper factors with the correct level of emphasis. Additionally, 
standardized and quantifiable measures must be implemented within DoD services 
contracting. These measures should align with S.M.A.R.T. metrics and incorporate a 
more objective orientation. This standard should incorporate only those definitions and 
measures that both support the objectives and goals of the DoD, as outlined in chapters I 
and II, and are validated by correlation in responses between stakeholders. Furthermore, 
the standard metrics incorporated should meet the criteria as outlined by S.M.A.R.T. We 
suggest utilizing a balanced outcome and process oriented approach, incorporating the 


following categories when defining a successful service contract: 
° schedule, maintain costs, communication and well defined requirements 


The standard measure for a successful service contract should incorporate solely 


an outcome driven approach utilizing the following three metrics: 


e performance, track costs, and track schedule 
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These standard metrics are aligned with the FAR’s determination that the 
customer should be satisfied in terms of cost, timeliness, and quality of the delivered 
product or service. Additionally, in accordance with the COR handbook, the COR and 
PCO must ensure that contract performance is timely and within scope of the work 
(Director, Defense Procurement and Acquisition Policy, 2012). Our recommended 
standards of defining and measuring the success of a service contact ensure adherence to 
these guidelines. These standards definitions and metrics for a successful service contract 
align with the goals and initiatives as outlined by the USD (AT&L), to properly manage 
and assess each service contract to determine its performance. Incorporation of these 
standards will allow for a more uniform analysis of this performance and could lead 
directly to the goals and objectives of the USD (AT&L), to promote efficient and 
effective services contracting, while reducing waste and producing savings. Additionally, 
this standard set of metrics and definitions will lead to better analysis on performance of 
a service contractor, leading to a selection of more efficiency and effective contractors in 


future programs. 


In addition, these standards for defining and measuring a successful service 
contract could be represented using a scorecard approach. The balanced scorecard is a 
planning and management tool within government organizations to align activities to the 
goals and objectives of the organization, improve communications, and monitor 
organization performance against goals. The scorecard employs a_ performance 
measurement framework added to financial metrics to give managers a complete view of 
organizational performance (Monczka, Handfield, Giunipero, & Patterson, 2011). 
Although important, cost is a lagging indicator of performance and not the only 
determining factor. The scorecard ensures other related factors (tracking schedule and 


performance) are given proper emphasis in evaluating service contract success. 


Incorporating these standardized definitions and quantifiable measurements in the 
form of a scorecard will mitigate the conflicts that arise due to the differing objectives 


and goals of the various stakeholders within DoD services contracting. 


The PM is an integral member of the IPT and an important stakeholder in the 


contracting process. Every service contract should be evaluated on complexity and 
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recognize when a need exists to incorporate a properly trained and assignable PM as a 
key stakeholder (Phillips, 2007). PMs provide strong communication skills and 


leadership to the entire team. 


An improved application of surveillance and past performance could be 
implemented. For example, the FAR should require a QASP for contracts below the 
simplified acquisition threshold. Incorporating a QASP will ensure oversight for the 
majority of service contracts. Even though the FAR 15.304 states that past performance 
evaluation should always be evaluated for negotiated competitive acquisitions expected 
to exceed the simplified acquisition threshold, CPARS is only required for acquisitions 
above the $1 million threshold. The CPARS threshold should be amended to include 
contracts below $1 million down to a minimum of the simplified acquisition threshold or 
at a point acceptable to capture the majority of service contract past performance data. In 
order to emphasize COR importance, individual contracting agencies should include 
instructions and directives mandating involvement of the COR in the pre-award phase 


activities, such as development of the SOW/PWS. 


J. SUMMARY 


In this chapter we presented and analyzed the data we collected from the research 


to answer the three research questions: 


1. How are successful service contracts within the DoD being defined by 


different stakeholders? 


2: How are service contracts being measured within the DoD by different 
stakeholders? 
3. How should service contracts be defined and measured within theDoD? 


We presented conclusions and recommendations based on our analysis. In the 
next chapter, we summarize our findings and provide recommendations for further 


research. 


54 


V. SUMMARY, CONCLUSION, AND AREAS FOR 
FURTHER RESEARCH 


A. SUMMARY 


Over the last few decades, a growing trend of increased dollar value in service 
contracts for Department of Defense (DoD) contracting has become apparent. Relative to 
supply contracts, services acquisition has continued to grow both in terms of dollar value 
and in range of acquisitions. Contract obligations rose to over $387 billion in 2008, with 
nearly $200 billion spent on services alone (Hutton & Solis, 2009). Funding spent on 
service contracts grew steadily from 1990-2010, constituting roughly 42% of the total 
spending on contracts by the DoD, exclusive of research and development services 
contracts. As such, a standardized approach to defining and measuring the success of 


services contracts is essential. 


The DoD IG and GAO have indicated poorly defined requirements and 
inadequate requirements management as a problem in service contracting (GAO, 2007b; 
DoD IG 2009) The GAO has described the DoD’s current approach to service contract 
management as reactive and not fully addressing the key factors of success (GAO, 
2007b). DoD IG and GAO reports have consistently identified issues in service contract 
administration and oversight (GAO, 2005, 2007a, 2007b; DoD IG, 2009). According to 
the GAO, the poor management of service contracts has undermined the government’s 
ability to obtain a good value for the money spent and contributed to the GAO’s decision 
to designate contract management a high-risk area for the DoD (GAO, 2001, 2007c, 
2011). 


The DoD implements cross functional teams through the use of an Integrated 
Product Team (IPT). It is useful to note that the IPT is primarily used in contracting for 
products to facilitate the process of meeting cost, performance, and schedule objectives 
from product concept through production, including field support (Press, 2012). In 
service contracting the IPT is a team composed of representatives from appropriate 


functional disciplines working together to identify and resolve issues; make sound, timely 
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recommendations in an effort to facilitate decision-making; and build successful 


programs that meet the warfighter’s needs (Press, 2012). 


With the differing goals, objectives, and responsibilities of the various 
stakeholders (PM, PCO, COR) within DoD services contracting, conflicts arise with 
regard to how a successful service contract is defined and measured between the 
stakeholders. This conflict is explained and understood through stakeholder theory. 

B. CONCLUSIONS 
i, Findings 
The research answered the questions contained in Chapter I: 


Le How are successful service contracts within the DoD defined by different 


stakeholders and what factors are considered in their definitions? 


2: How are service contracts being measured within the DoD by different 
stakeholders? 
3. How should service contracts be defined and measured within the DoD? 


We found that there is no standardized definition or measurements for success of 
service contracts. While similarities do exist between the definitions and measures of 
success of a service contract, the weighted value of each factor and metric varies between 
the stakeholders. Many of the metrics that were established failed to properly address the 
characteristics of S.M.A.R.T. either partially or entirely. 


We discovered that the differing objectives and roles of each stakeholder clearly 
affected the factors that each considered when defining and measuring the success of 
service contracts. Our research revealed that CORs and PCOs define the factors of a 
successful service contract as the following: schedule, maintain costs, communication, 
and well defined requirements. Additionally, we discovered CORs and PCOs similarly 
measure a successful service contract as the following: performance, track costs, 


customer satisfaction and end user evaluation and track schedule. 
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Further, we discovered a lack of properly established PMs within DoD services 
contracting, with contributes to the deficiencies already present in services contracting. 
Additionally, we discovered that no standardized certification process was established 


and required for CORs within services contracting. 


We also discovered the COR is not always involved in the pre-award phase of 


service contracts and current application of QASP and CPARS is inadequate. 


2. Recommendations 


A standardized definition of a successful service contract must be implemented, 
incorporating the proper factors with the correct level of emphasis. Additionally, 
standardized and quantifiable measures must be implemented within DoD services 
contracting. These measures should align with S.M.A.R.T. metrics and incorporate a 
more objective orientation than currently that which currently exists. Incorporating these 
standardized definitions and quantifiable measurements will mitigate the conflicts that 
arise due to the differing objectives and goals of the various stakeholders within DoD 
services contracting. Every service contract should be evaluated on complexity and 
evaluators should recognize when a need exists for a PM. An improved application of 
surveillance and past performance should be implemented. In order to emphasize COR 
importance, individual contracting agencies should include instructions and directives 
mandating involvement of the COR in the pre-award phase activities, such as 


development of the SOW/PWS. 


In addition, we suggest utilizing a balanced outcome and process oriented 
approach, incorporating the following categories when defining a successful service 
contract: schedule, maintain costs, communication and well defined requirements. The 
standard measure for a successful service contract should incorporate solely an outcome 
driven approach utilizing the following three metrics: performance, track costs, and track 
schedule. These standards for defining and measuring a successful service contract could 
be represented using a balanced scorecard approach. The balanced scorecard is a 
planning and management tool within government organizations to align activities to the 


goals and objectives of the organization, improve communications, and monitor 
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organization performance against goals. Incorporating these standardized definitions and 
quantifiable measurements in the form of a scorecard will mitigate the conflicts that arise 
due to the differing objectives and goals of the various stakeholders within DoD services 


contracting. 


C: AREAS FOR FURTHER RESEARCH 


Our research participants consisted solely of commands associated with the 
United Sates Navy. As such, we recommend continuing this research into the various 
contracting commands within the United States Air Force, Army, and Marine Corps. We 
were also able to incorporate very little data from PMs and contractors within DoD 
services contracting. We recommend that future research incorporate a larger portion of 
PMs and contractors. Additionally, we recommend that future research incorporate 
greater use of the survey tools, such as Survey Money, due to its ability to reach a larger 


audience more effectively and efficiently. 
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